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ABSTRACT 

We propose an adaptive affine combination of two adaptive filters in that combination select one is fast and one 
slow. In this paper we proposed LMS, NMLS and CM A algorithm technique. By using these algorithm 
techniques we calculate the mixing parameter (r|) at every instant and the performance of the measurement 
parameter that is excess mean square error EMSE is varied with the step size according to the variation the 
step size we achieve the good convergence rate and its adaptation is also taken into account in the transient 
analysis and steady state analysis. The proposed combination should acquire the good convergence properties 
for all kinds of stationary and non stationary environments. The resulting combination should profit than 
single filter technique. 

Keywords: Adaptive filters, transient analysis, steady state analysis, EMSE, CM A. 



I. INTRODUCTION 

One of the most popular algorithms for adaptive 
filtering is the LMS algorithm. LMS adjusts the 
adaptive filter weights and modifying them by an 
amount proportional to the instantaneous estimate of 
the gradient of the error surface [1]. It neither 
requires correlation function and matrix inversions 
method, which makes it simple and easy when 
compared to other algorithms. 

Minimization of MSE is achieved due to the iterative 
procedure incorporated in it[2].To make successive 
corrections in the direction of negative of the 
gradient vector of it .The adaptive filters that exhibit 
good convergence properties in stationary 
environments [3], do not necessarily present good 
tracking performance in non stationary environment, 
for combination of algorithms we calculate the 
convergence rate and EMSE in adaptive filters. 

The rest of the paper is organized as follows In the 
next section II, we describe the affine combination of 
two adaptive filters. In Section III, analytical 
expressions for the optimum mixing parameter and 
the optimum EMSE, Section IV will discuss about the 



steady-state analysis two LMS filters, two NLMS 
filters and two CM A equalizers. Section V is about 
simulation results and discussion, some conclusion is 
given in Section VI. 

II. COMBINATION OF SUPERVISED ALGORITHMS 

The linear combination of two supervised adaptive 
filters is depicted in Fig.l, where the filter weights 
are adjusted to minimize the mean-square error cost 
function, obtaining at the output an estimate of the 
given "desired signal" d(n). The output of the overall 
filter is given by 

y(n) = r/(n)y^(n)+{-r/(n)^2(^) (1) 

where T](n) is the mixing parameter and yi(n), i=l,2 
are the outputs of two transversal filters, 
i.e., y.(n) = u^ (n)wi (n - 1) .The superscript T denotes 

transposition, Wi(n - 1), \ = 1,2 represent the length-M 
coefficient column-vectors characterizing the 
component filters, and u(n) is their common input 
regressor column-vector 
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?](n + 1) = ?](n) + jUn^(n) f j (n) - y2 (n) _. 



.(6) 



Figl. Linear combination of two supervised adaptive 
filters 

We focus on the affine combination of two 
algorithms of the following general class 



Wi in) = Wi - 1) + Pi (n)u(n)ei (n) 



(2) 



where pi(n) is a step-size and ei(n) is the estimation 
error. Many algorithms can be written, by proper 
choices of pi(n) and ei(n). In supervised adaptive 
filtering, a "'desired signal" d(n) is available such 



e: in) = din) - y: (n) 



(3) 



and a linear regression model holds, i.e.. 



T 

d(n) = u (n)w^(n — l) + v(n) 



(4) 



with Wo(n- 1) being the time- variant optimal solution 
and v(n) a zero-mean random process uncorrelated 
with u(n), whose variance is denoted by 
a^=Ex^(n) the sequences {u(n)} and {v(n)} are 

assumed stationary and that v(n) is independent of 
u(n) (not only uncorrelated). Defining the weight 

error Vectors Wi (n) = (n) - (n) the a priori errors 



.(5) 



An important consequence of this model is that v(k) 

will be independent of all Wiij), Wi (j) and ea,i(k) , i = 
1,2 j < k. 

For any particular time instant k. considering the 
combination of two LMS filters and the minimization 
of the overall instantaneous square error 

e^(n)= f (n)- ^proposed the following gradient 

based algorithm 



To obtain a tradeoff between stability of this 
recursion and the algorithm's tracking capability in 
the initial phase of adaptation, rjin) <=1 

A) Combination of blind algorithms 
It is a simplified communications system with a 
combination of two blind equalizers. In this case, the 
signal a(n), assumed i.i.d. (independent and 
identically distributed) and non Gaussian, is 
transmitted through an unknown channel, whose 
model is constituted by an FIR filter and additive 
white Gaussian noise.. 

Algorithms based on the constant modulus cost 
function define the "estimation error" as 



(7) 



where r = E{aHn)} I E{aHn)} . 



These assumptions were used in[10] obtain simple 
linear models that capture the behavior of CMA close 
to an optimum solution. 

Thus, (7) was approximated by 
ei(n)^y(n)ea,i(n)+p(n ... .(8) 

where 

y(n)=3aHntd)r (9) 

and 

P(n)=ya(ntd)a^(ntd) (10) 

The variable ^(n) is identically zero for constant- 
modulus constellations, so the variability in the 
modulus of a(n) (as measured by ^(n)) plays the role 
of measurement noise for constant-modulus based 
algorithms. Model [5] was proposed to study convex 
combinations of constant-modulus based algorithms 
and extended to obtain explicit stability conditions 
for CMA to update the mixing parameter in order to 
combine two CMA equalizers, we could use a 
gradient rule to minimize the instantaneous 

[2 2 
-y (n) as 

considered in the convex combination.. Thus, we 
propose a stochastic gradient algorithm to minimize 
the instantaneous square decision error 

Jd{n) = el in), 
where 

A A A 

(n) = a(n -t^)- y(n) and a(n - ) 
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the estimate of the transmitted signal at the output of 
the decision device. This results in the following 
update equation 

7j(n + 1) = 7j(n) + ju^e^ (n) (n) - ^2 W _ • ■ • .(H) 

In the presence of noise and/or when both 
component filters are far from convergence, assume 

ed(n) ^ea(n) 

B) A common formulation 

Comparing (8) to (5), we can write the following 
general expression 



eg(n) =ea(n) + b(n) 



(15) 



ei(n)=k(n)ea,i(nh(p(n) ,i=l,2.. 



(12) 



where k = 1 and (p(n) = v(n) for a supervised 
algorithm or K(n) = y(n) and (p(n) = ^(n) for a blind 
one. In both cases E{ rjin)} = 0. This model also holds 
for the overall scheme, i.e.. 



and b(n) = v(n) for the combination of supervised 
algorithms or b(n) = 0 for the combination of 
constant-modulus-based algorithms. In both cases, 
ri(n) is constrained to be less than or equal to 1 for all 
n . Algorithm (14) is denoted here by r|-LMS. 

III. THE OPTIMUM MIXING PARAMETER AND EMSE 

An analytical expression for the optimum mixing 
parameter rjoin) can be obtained equating to zero the 
expected value of the gradient used to update rjin) in 
(14), i.e.. The error eg(n) in (16) can be rewritten as a 
function of the a priori errors ea,i(n), 

i = 1, 2, as follows. Using (1), (12), and (13), the a 
priori error ea(n) of the overall scheme can be written 
as 

M = M + [1 - ^(n)]ea,2 W ^^^^ 

= ea,2 M + Veax (n) - ^^,2 (2)] 



e(n)=K(n)ea(nh rjin) 



13) 



where e(n) represents the error of the combined 
filter: e(n)=d(n)-y(n) for supervised algorithms or e(n) 
= [r - y^(n)]y(n) for constant-modulus-based 
algorithms, and ea(n) is the a priori error of the 
overall scheme. It should be noticed that (12) and (13) 
are approximations in the blind case. For the sake of 
simplicity, we use the equality sign here and in the 
expressions derived from (12) and (13). 

The supervised LMS and NLMS algorithms and the 
blind CMA employ the step-sizes Qi(n)and the 
estimation errors e/fn), where Qn is a regularization 
factor and k. k represents the Euclidean norm. The 
models for the errors ei(n) of these algorithms are also 
shown in this table for convenient reference. The 
step-size interval which ensures the convergence and 
stability is different for each algorithm. For the LMS 
and NLMS algorithms, the step -size intervals are 
well-known in the literature whereas for CMA, the 
derivation of this interval was shown recently. 

Using equation model (5) in the supervised case, and 
the fact that ed(n) ^ ea(n) in the blind case, we can 
write a general expression for updating the mixing 
parameter, i.e., 

?](n+l )= r](n)+ jUnCgin) [yi(n)y2(n)] (14) 

where 



Replacing (17) in (15), and remarking that 

yi(n)-y2(n)=ea,i(n)-ea,2(n) and 



...(18) 



+ 



In the blind case, h(n) = 0 and in the supervised case, 
b(n) = v(n), which is assumed independent of ea,i(n), i 
= 1, 2. Hence, in both cases the third term on the 
L.H.S. of (18) is equal to zero. 

To proceed, we remark that the EMSE of the 
component filters and the cross-EMSE can be 
calculated (18), respectively as 



gii{n)-E ^-{n)^ / = 1,2 and 



1 
J 



(19) 



(20) 



Introducing the differences 

ACn(n) = Cn(n) - C^2(nh i = 1,2, (21) 

and using (19)-(21) in (18), we arrive at 

A^22 (^n) 



Ag^^(n) + Ag22W 



(22) 



A similar expression was also obtained for the 
convex combination of two LMS filters at the steady- 
state. We should notice that (22) is more general: it 
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holds for all n > 0 (not only at the steady-state) and 
the mixing parameter is not restricted to the interval 
[0, 1]. Defining the EMSE of the overall combined 
scheme as 

C(n)=E{e^a(n)} (23) 

We now obtain an analytical expression for its 
optimum value. By squaring both sides of (17) with 
ri(n) = rio(n) and taking expectations, we arrive at 



In this model, q(n) is an i.i.d (independent and 
identically distributed), vector with positive-definite 
autocorrelation matrix Q=E{q(n)q^(n)} independent 
of the initial conditions {wo(-l),w(-l),ri(-l)} and of 
{u(\)} for all 1 [4]. In supervised filtering, q{n) is also 
assumed independent of the desired response {d{\)} 
for all 1 < n. In blind equalization, Wo{n) represents the 
zero-forcing solution and q{n) models the channel 
variation. The analyses. 



E{e^a(n)} = jfo (n) E{e^a,i(n)} 

+ 11- r]o(n)?E{e\2(n) 
+2r]o(n)[l-r]o(n)]E{ea,i(n)ea,2(n)} 



.(24) 



Using (19)-(22) in (24), we obtain 

(^o(n) = ^iiin) - rjo MA^iiin) 



,(25) 



After some algebraic manipulations, (25) can be 
rewritten as 



(26) 



This expression was obtained for the convex 
combination of two LMS filters at the steady state, 
but again it also holds for all n > O.As already 
mentioned in (8) (22) (26) hold for the combination of 
any two algorithms that satisfy (12). The values of 
A(^ii(n) ,i = 1,2 however do depend on the actual 
algorithms that are being combined .Thus provided 
approximations for (^ii(n) i = 1,2 are available, (22) 
and (26) can applied to the affine combination of 
different algorithms ,including combinations of 
algorithms of different families. 

IV. STEADY-STATE ANALYSIS OF THE OPTIMUM 
COMBINER 

In this section, the optimum mixing parameter and 
the optimum EMSE of the combination, given 
respectively by expressions (22) and (26), are 
particularized for the combination of two LMS filters, 
two NLMS filters, and two CMA equalizers in 
steady-state for stationary and non stationary 
environments. We assume that in a non stationary 
environment, the variation in the optimal solution 
wo follows a random- walk model, that is. 



wo(n) =Wo(n -1) + q(n) 



(27) 



A) Transient Analysis of Realizable Schemes 
In this section, we take into account the adaptation of 
r\{n) in the analysis By squaring both sides of (17) and 
taking expectations, we Obtain 

(29) 

To proceed, we assume that: 

Al. The adaptation of rjin) is slow so that the 
correlation between it and ea,i(n),ea,i(n)J,i = 1,2 
can be disregarded. 

This assumption follows from observations: 
simulations show that iqin) converges slowly 
compared to variations in the input u(n) and thus to 
variations on the a-priori errors. Using Al, (19)-(21) 
and (23), we can rewrite (29) as 

gin) « ^22 (^) + W ^(^) 2£' f^in) ^^22(^) 

(30) 

where we define 

a(n) = ^ li (n) - y 2 (n) ^ j A^i ^ (n) + A^22 M 

(31) 



To estimate the EMSE of the combination for all n >0 
using (30), analytical expressions for ^ijin), i = 1,2, 
E{r](n)}, and E{rf(n)} should be obtained. It is 
common in the literature to evaluate the EMSE as 

^//(^) = ^4,/(^K,y (^) jTr{RSij{n-l)) ....(32) 



V. SIMULATION RESULTS 

To verify the transient analysis in the supervised 
case, we consider the identification of a time 
invariant system. The optimum solution is formed 
with M = 7 independent random values between -1 
an 1, and is given by Wo = [+0.90 -0.54 -0.03 +0.78 
+0.52 -0.09] .We assume white Gaussian input with 
variance 1/M so that Tr(R) = 1, and an average of 500 
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runs. Moreover, i.i.d. noise v (n) with variance v 
= 0.01 is added to form the desired signal. 

Fig. 2 shows the results of the EMSE and the mixing 
parameter for the affine combination of two LMS 
filters in the same situations considered in Fig. 3 and 
4 in which the mixing parameter is updated with the 
N-LMS algorithm, where |an = 3,. Similarly, with |an = 
0.1, the analysis can predict that the combination is 
not able to switch to the slow filter. We should notice 
that, due to the constraint imposed in the r|-LMS 
algorithm (r|(n) < 1),. 

plot for un = 0.1 
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Fig. 2. Affine combination of two signals. 
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Fig. 3. Ensemble average of n(n) adapted with the N- 
LMS algorithm. 



V 


L 












I I 

-e- Eg from (26) 












































\ 






















\ 


































































) € 


) ( 


) i 


) i 


5 0 



0 1 2 3 4 5 6 7 8 9 10 

iterations 



Fig. 4. Convex combination of two LMS filters at the 
steady state, but it holds for all n > 0. 
VI. CONCLUSIONS 

We proposed transient and steady-state analyses for 
the EMSE and the mixing parameter of the affine 
combination, based on the theoretical EMSE and 
cross-EMSE of the component filters and on the 
adaptation of the mixing parameter. This states the 
application to different combinations of algorithms 
of LMS, NLMS and CMA, considering white or 
colored inputs and stationary or non stationary 
environments. Good agreement between the analysis 
and the simulations was always observed. Moreover, 
we proposed and analyzed two normalized 
algorithms for updating the mixing parameter. The 
theoretical models can predict situations in which 
these algorithms can achieve a good performance 
results. 
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